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Description 

[0001] The present invention. relates to a self-service 
terminal (SST). In particular, the invention relates to a 
public access self-service terminal such as an automat- 
ed teller machine (ATM) or a non-cash kiosk. 
[0002] It is well known that ATMs are commonly used 
as a convenient source of cash and other financial trans- 
actions. Some users of ATMs desire a quick cash dis- 
pense transaction (sometimes referred to as fast cash) 
without viewing any promotional material (such as ad- 
vertisements ormarketing information) or other services 
(such as other transactions). Other users, however, are 
willing to view promotional material and/or services on 
ATMs depending on certain factors, such as whether 
they are in a hurry, whether they are interested in the 
type of product or service that is being promoted, or such 
like. If promotional material or services are presented to 
users at inappropriate times, or if an ATM transaction is 
lengthened because of presenting other services or so- 
liciting some input from the user, then the user may be 
annoyed by the delay and as a result may be unsatisfied 
with the ATM transaction. 

[0003] It is among the objects of an embodiment of 
the present invention to obviate or mitigate one or more 
of the above disadvantages or other disadvantages as- 
sociated with prior art self-service terminals. 
[0004] According to a first aspect of the present inven- 
tion there is provided a self-service terminal having a 
user interface for interacting with a user, characterised 
in that the terminal includes sensing means for sensing 
physiological data associated with a user, analysing 
means for analysing the physiological data to deduce 
the user's emotional state, and control means respon- 
sive to the analysing means for adapting the terminal's 
interaction with the user in response to the user's emo- 
tional state. 

[0005] The sensing means may be implemented us- 
ing a contact device, but more preferably, using a non- 
contact device. 

[0006] Where a contact device is used, the sensing 
means may comprise a touch area incorporating sen- 
sors for determining the user's skin temperature, pulse 
rate, blood pressure, skin conductivity, and such like 
physiological data. A suitable touch area may be imple- 
mented by a device developed by IBM (trade mark) and 
called an "emotion mouse". 

[0007] Where a non-contact device is used, and the 
terminal includes speech input, the sensing means may 
be implemented by a voice monitoring system for de- 
tecting changes in a user's voice. Additionally or alter- 
natively, the sensing means may be implemented by fa- 
cial recognition to detect changes in the user's facial ap- 
pearance during a transaction. The sensing means may 
be implemented using an iris camera for imaging the us- 
er's iris and for detecting changes within the iris, such 
as changes to blood vessels, and such like. The sensing 
means may be implemented by a gesture recognition 
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system. 

[0008] The analysing means may be implemented by 
any convenient algorithm for deducing a person's emo- 
tional state from physiological measurements taken 

5 from the person . An overview of such algorithms is given 
in chapter 6 of "Affective Computing" by Rosalind W 
Picard, MIT Press, 1997, ISBN 0-262-16170-2. 
[0009] The control means may be implemented by a 
control application executing on the terminal. The con- 

10 trol application may present a user with a sequence of 
screens to guide the user through a transaction. The 
control application may determine which screens are to 
be shown to the user in response to the user's emotional 
state as deduced by the analysing algorithm. 

is [001 0] The term "screen" is used herein to denote the 
graphics, text, controls (such as menu options), and 
such like, that are displayed on an SST display; the term 
"screen" as used herein does not refer to the hardware 
(for example, the LCD, CRT, or touchscreen) that dis- 

20 plays the graphics, text, controls, and such like. Typical- 
ly, when a transaction is being entered at an SST, a se- 
ries of screens are presented in succession on the SST 
display. For example, a first screen may request a user 
to insert a card, a second screen may invite the user to 

25 enter his/her personal identification number (PIN), a 
third screen may invite the user to select a transaction, 
and so on. 

[001 1 ] By virtue of this aspect of the invention, the ter- 
minal is able to sense physiological data from a user 

30 during a transaction, analyse the data, and determine 
what to present to the user to comply with the user's 
emotional state. For example, the terminal may deter- 
mine what transaction options to present, whether to 
present advertising ormarketing material, if advertising 

35 is to be presented then what advertisements to present, 
for how long the advertisement is to last, at what point 
in the transaction the advertisement is to be shown, and 
such like. 

[0012] Thus, the user's experience at the SST can be 
40 improved by personalising a transaction to the user's 
emotional state. For example, if a user feels insecure 
then the SST may: 

• highlight to the user alternative SST locations at 
45 which they user may feel more secure; 

• display a message regarding privacy and trust to 
reassure the user that the transaction is secure and 
that the transaction provider is one that the user can 

50 trust to keep user information private; 

• give the user an option of more time to select a 
transaction option. 

55 [0013] The SST can improve the transaction provid- 
er's marketing and advertising efficiency by targeting 
advertisements that are known to be more effective for 
a particular emotional state. For example, if the user is 
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in a happy mood and relaxed then the user might be 
receptive to advertising and the SST may: 

• present humorous advertisements, and/or 

• have a longer transaction sequence to provide 
more advertising time. 

[0014] The SST may record a user's emotional expe- 
rience so that future transactions conducted by that user 
are automatically personalised. For example, if the user 
is not in a relaxed mood then the user might be irritated 
by advertising and the SST may: 

• adapt the transaction to have no advertising during 
that transaction, or 

• go to a customised quick transaction flow for that 
user the next time they use the SST 

[0015] If the SST detects particular emotional states 
of users, then the SST may invoke extra security meas- 
ures to improve security for both the users and the SST 
provider. For example, if the user was detected as being 
under a great deal of stress then this may indicate that 
the user is executing a transaction under duress, or the 
user may be using a stolen transaction token (such as 
a magnetic stripe card). These extreme types of emo- 
tional states could trigger additional security measures 
at the SST, such as: 

• more security photographs being taken; and/or 

• security information being requested from the user. 

[0016] In one embodiment the SST is an ATM. 
[0017] According to a second aspect of the present 
invention there is provided a method of operating a self- 
service terminal, the method comprising the steps of: 
sensing physiological data associated with a user of the 
terminal, analysing the physiological data to deduce the 
user's emotional state, and adapting the terminal's in- 
teraction with the user in response to the user's emo- 
tional state. 

[001 8] These and other aspects of the present inven- 
tion will be apparent from the following specific descrip- 
tion, given by way of example, with reference to the ac- 
companying drawings, in which: 

Fig 1 is a block diagram of a self-service terminal 
according to one embodiment of the present inven- 
tion; 

Fig 2 is a block diagram of a part (the controller) of 
the terminal of Fig 1 ; 

Figs 3A to 3F illustrate a sequence of screens pre- 
sented to one user of the terminal of Fig 1 ; and 
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Figs 4A to 4H illustrate a sequence of screens pre- 
sented to another user of the terminal of Fig 1 . 

[0019] Reference is now made to Fig 1, which illus- 
5 trates an SST 1 0 in the form of an ATM being operated 
by a user 1 2. 

[0020] The ATM 10 includes a user interface 14 for 
outputting information to a user and for allowing a user 
to input information. The ATM 10 also includes sensing 
10 means 16 in the form of a camera module 18 (that in- 
cludes facial recognition software), atouch plate module 
20 (implemented by an "emotion mouse"), and a micro- 
phone module 22 (that includes voice recognition soft- 
ware). 

15 [0021] The user interface 14 is a moulded fascia in- 
corporating: a display module 30, an encrypting keypad 
module 32, and a plurality of slots aligned with modules 
located behind the fascia. The slots include a card entry/ 
exit slot (not shown) that aligns with a magnetic card 

20 reader/writer (MCRW) module 36, a printer slot (not 
shown) that aligns with a printer module 38, and a cash 
dispense slot (not shown) that aligns with a cash dis- 
pense module 40. 

[0022] The ATM 1 0 also includes an internal journal 
25 printer module 50 for creating a record of all transactions 
executed by the ATM 1 0, an ATM controller module 52 
for controlling the operation of the various modules (18 
to 50), and a network connection module 54 for commu- 
nicating with a remote transaction host (not shown) for 
30 authorising transactions. All of the modules (18 to 54) 
within the ATM 1 2 are interconnected by an internal bus 
56 for securely conveying data. 

[0023] The ATM controller module 52 is shown in 
more detail in Fig 2. The controller 52 comprises a BIOS 

35 60 stored in non-volatile memory, a microprocessor 62, 
associated main memory 64, and storage space 66 in 
the form of a magnetic disk drive. " 
[0024] In use, the ATM 12 loads an operating system 
kernel 70, an ATM application program 72, and a data 

40 analysing program 74 into the main memory 64. 

[0025] The ATM application program 72 is used to 
control the operation of the ATM 12. In particular, the 
ATM application program 72: provides the sequence of 
screens used in each transaction (referred to as the ap- 

45 plication flow); monitors the condition of each module 
within the ATM (state of health monitoring); and inter- 
faces with the analysing program 74. 
[0026] The analysing program 74 implements a dis- 
criminant function analysis model for analysing data re- 

50 ceived from the sensor modules 1 8 to 22; however, any 
other convenient analysing program may be used. The 
analysing program 74 processes data received from 
one or more of the sensor modules (camera 18, touch 
plate 20, or microphone 22) to deduce the emotional 

55 state of the user 1 2. 

[0027] The analysing program 74 selects an emotion 
category that is the closest match to the user's emotion- 
al state, and outputs a code representing this category 
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to the ATM application program 72. In this embodiment, 
the categories are: anger, hurriedness, fear, happiness, 
sadness, and surprise. 

[0028] The ATM application program 72 receives this 
code and adapts the transaction flow according to the 
emotional state represented by this code. This is imple- 
mented by the ATM application program 72 accessing 
a stored look-up table (not shown) having an index entry 
for each code. Each code in the look-up table has a 
unique transaction flow associated with it. 
[0029] An example of a typical transaction at the ATM 
10 will now be described with reference to Figs 3A to 
3F, which illustrate the sequence of screens presented 
to the user 12. 

[0030] When the user 12 approaches the ATM 1 0 he 
is presented with a welcome screen 80a (Fig 3A) on dis- 
play 30 inviting him to insert his card. After inserting his 
card, the user 12 is presented with a screen 80b (Fig 
3B) inviting him to enter his PIN, and the ATM applica- 
tion program 72 activates the sensors 1 8 to 22 to cap- 
ture physiological data about the user 12. 
[0031] The ATM application receives data from the 
sensors 18 to 22 and conveys this data to the data an- 
alysing program 74. Data analysing program 74 proc- 
esses the received data, deduces the user's emotional 
state from the data, generates a category code repre- 
senting the user's emotional state, and conveys this 
code to the ATM application program 72. The ATM ap- 
plication program 72 accesses the look-up table (not 
shown) using the category code received from the data 
analysing program 74 to determine what sequence of 
screens should be presented to the user 12. In this ex- 
ample, the user's state is hurriedness, so the sequence 
of screens is that for the shortest possible transaction 
time. 

[0032] The ATM application program 72 then 
presents the user 12 with a screen 80c (Fig 3C) listing 
transaction options available. After the user 12 has se- 
lected the withdraw cash option, the ATM application 72 
presents the user with a screen 80d (Fig 3D) indicating 
cash amounts available. Once the user has selected a 
cash amount, the ATM application authorises the trans- 
action, presents a screen 80e (Fig 3E) inviting the user 
to remove his card, then a screen 80f (Fig 3F) inviting 
the user to remove the requested cash. 
[0033] An example of a typical transaction at the ATM 
10 will now be described with reference to Figs 4A to 
4H, which illustrate the sequence of screens presented 
to another user (or the same user as for Figs 3A to 3F 
but in a different emotional state). 
[0034] When the user approaches the ATM 10 he is 
presented with a welcome screen 82a (Fig 4A) on dis- 
play 30 inviting him to insert his card. After inserting his 
card, the user is presented with a screen 82b (Fig 4B) 
inviting him to enter his PIN, and the ATM application 
program 72 activates the sensors 18 to 22 to capture 
physiological data about the user. 
[0035] As in the previous example, the ATM applica- 



tion 72 receives data from the sensors 1 8 to 22 and con- 
veys this data to the data analysing program 74. Data 
analysing program 74 processes the received data, de- 
duces the user's emotional state from the data, gener- 

5 ates a category code representing the user's emotional 
state, and conveys this code to the ATM application pro- 
gram 72. The ATM application program 72 accesses its 
look-up table (not shown) using the category code re- 
ceived from the data analysing program 74 to determine 

10 what sequence of screens should be presented to the 
user. In this example, the user's state is happiness, so 
the sequence of screens includes an advertisement for 
a holiday, and promotional material for a loan. 
[0036] The ATM application program 72 then 

15 presents the user with a screen 82c (Fig 4C) listing 
transaction options available. After the user has select- 
ed the withdraw cash option, the ATM application 72 
presents the user with a screen 82d (Fig 4D) indicating 
cash amounts available. 

20 [0037] Once the user has selected a cash amount, the 
ATM application 72 authorises the transaction, and 
presents the user with a screen 82e (Fig 4E) incorpo- 
rating a video 84 (in M PEG format) advertising a holiday, 
the screen 82e also includes text 86 informing the user 

25 that the requested transaction is being authorised. 

[0038] Once the video (which lasts approximately four 
seconds) has finished, the ATM application 72 then 
presents the user with a screen 82f (Fig 4F) incorporat- 
ing promotional material 88 for a loan. 

30 [0039] The ATM application 72 then presents a 
screen 82g (Fig4G) inviting the user to remove his card, 
and once the card has been removed, a screen 82h (Fig 
4H) inviting the user to remove the requested cash. 
[0040] It will be appreciated that this embodiment has 

35 the advantage that a user is presented with a transaction 
sequence that is most likely to fulfil the user's expecta- 
tions by matching a transaction to the user's emotional 
state. 

[0041] Various modifications may be made to the 

40 above described embodiment within the scope of the in- 
vention, for example, in other embodiments, the user 
may be asked to touch the touch plate 20 at the begin- 
ning of the transaction so that the touch plate can collect 
physiological data from the user's hand. In other embod- 

45 iments, multiple algorithms may be used to implement 
the analysing program 74, one for each sensor module 
18 to 22. In other embodiments, different sensors may 
be used. In other embodiments, the touch plate sensor 
may be implemented on the keys of the encrypting key- 

50 pad so that physiological measurements can be taken 
while the user is entering his PIN or other transaction 
details. In other embodiments, the user's emotional 
state may be continually monitored during the transac- 
tion flow so that the transaction flow may be changed at 

55 any point in response to the user's emotional state; for 
example, an advertisement may be stopped if a user's 
emotional state changes from being happy or relaxed to 
being unhappy or angry. In other embodiments, different 



4 



7 



EP 1 246 146 A2 



8 



emotional states may be categorised than those de- 
scribed in the above embodiment. 



Claims 

1 . A self-service terminal (1 0) having a user interface 
(14) for interacting with a user (12), characterised 
in that the terminal (10) includes sensing means 
(18, 20, or 22) for sensing physiological data asso- 
ciated with a user, analysing means (74) for analys- 
ing the physiological data to deduce the user's emo- 
tional state, and control means (72) responsive to 
the analysing means (74) for adapting the terminal's 
interaction with the user in response to the user's 
emotional state. 

2. A terminal according to claim 1, wherein the sensing 
means is implemented using a contact device (20). 

3. A terminal according to claim 2, wherein the sensing 
means comprises a touch area incorporating sen- 
sors for determining any two types of physiological 
data from the following list: skin temperature, pulse 
rate, blood pressure, and skin conductivity. 

4. A terminal according to claim 1 , wherein the sensing 
means is implemented using a non-contact device 
(18 or 22). 

5. A terminal according to claim 4, wherein the termi- 
nal includes speech input, and the sensing means 
is implemented by a voice monitoring system (22) 
for detecting changes in a user's voice. 

6. A terminal according to claim 4 or 5, wherein the 
sensing means is implemented by facial recognition 
to detect changes in the user's facial appearance 
during a transaction. 

7. A terminal according to claim 6, wherein the sensing 
means is implemented using a human iris camera 
(18) for imaging the user's iris and for detecting 
changes within the iris. 

8. A terminal according to any of claims 4 to 6, wherein 
the sensing means is implemented by a gesture 
recognition system. 

9. A terminal according to any preceding claim, where- 
in the analysing means (74) is implemented by an 
algorithm for deducing a person's emotional state 
from physiological measurements taken from the 
person. 

1 0. A terminal according to any preceding claim, where- 
in the control means (72) is implemented by a con- 
trol application executing on the terminal. 



11. A method of operating a self-service terminal, the 
method comprising the steps of sensing physiolog- 
ical data associated with a user of the terminal, an- 
alysing the physiological data to deduce the user's 
5 emotional state, and adapting the terminal's inter- 

action with the user in response to the user's emo- 
tional state. 
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WELCOME 
PLEASE ENTER YOUR CARD 



80a 




Fig 3A 



PLEASE ENTER 
YOUR PIN 



80b 



Fig 3B 



PLEASE SELECT TRANSACTION- ^ 

• CASH 

• TRANSFER FUNDS 

• CHEQUE 

• CANCEL 
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80d 



Fig 3D 



PLEASE REMOVE 
YOUR CARD 



Fig 3E 



PLEASE REMOVE 
YOUR CASH 



80e 



80f 



Fig3C 



Fig 3F 
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WELCOME 
PLEASE ENTER YOUR CARD 



82a 



86 



AUTHORISING TRANSACTION 



Holiday? 



ft 



82e 



84 



Fig 4A 



Fig4E 



PLEASE ENTER 
YOUR PIN 



82b 



AUTHORISING TRANSACTION^ - 
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o^T 82f 



Need a loan to fund 
your dream holiday? 
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Fig4B 



Fig4F 
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• CASH 

• TRANSFER FUNDS 

• CHEQUE 

• CANCEL 
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PLEASE REMOVE 
YOUR CARD 



82g 



Fig 4C 



Fig 4G 
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PLEASE REMOVE 
YOUR CASH 



82h 



Fig4D 
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